195 research outputs found
A review of the literature on citation impact indicators
Citation impact indicators nowadays play an important role in research
evaluation, and consequently these indicators have received a lot of attention
in the bibliometric and scientometric literature. This paper provides an
in-depth review of the literature on citation impact indicators. First, an
overview is given of the literature on bibliographic databases that can be used
to calculate citation impact indicators (Web of Science, Scopus, and Google
Scholar). Next, selected topics in the literature on citation impact indicators
are reviewed in detail. The first topic is the selection of publications and
citations to be included in the calculation of citation impact indicators. The
second topic is the normalization of citation impact indicators, in particular
normalization for field differences. Counting methods for dealing with
co-authored publications are the third topic, and citation impact indicators
for journals are the last topic. The paper concludes by offering some
recommendations for future research
F1000 recommendations as a new data source for research evaluation: A comparison with citations
F1000 is a post-publication peer review service for biological and medical
research. F1000 aims to recommend important publications in the biomedical
literature, and from this perspective F1000 could be an interesting tool for
research evaluation. By linking the complete database of F1000 recommendations
to the Web of Science bibliographic database, we are able to make a
comprehensive comparison between F1000 recommendations and citations. We find
that about 2% of the publications in the biomedical literature receive at least
one F1000 recommendation. Recommended publications on average receive 1.30
recommendations, and over 90% of the recommendations are given within half a
year after a publication has appeared. There turns out to be a clear
correlation between F1000 recommendations and citations. However, the
correlation is relatively weak, at least weaker than the correlation between
journal impact and citations. More research is needed to identify the main
reasons for differences between recommendations and citations in assessing the
impact of publications
Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus
Journal classification systems play an important role in bibliometric
analyses. The two most important bibliographic databases, Web of Science and
Scopus, each provide a journal classification system. However, no study has
systematically investigated the accuracy of these classification systems. To
examine and compare the accuracy of journal classification systems, we define
two criteria on the basis of direct citation relations between journals and
categories. We use Criterion I to select journals that have weak connections
with their assigned categories, and we use Criterion II to identify journals
that are not assigned to categories with which they have strong connections. If
a journal satisfies either of the two criteria, we conclude that its assignment
to categories may be questionable. Accordingly, we identify all journals with
questionable classifications in Web of Science and Scopus. Furthermore, we
perform a more in-depth analysis for the field of Library and Information
Science to assess whether our proposed criteria are appropriate and whether
they yield meaningful results. It turns out that according to our
citation-based criteria Web of Science performs significantly better than
Scopus in terms of the accuracy of its journal classification system
Systematic analysis of agreement between metrics and peer review in the UK REF
When performing a national research assessment, some countries rely on
citation metrics whereas others, such as the UK, primarily use peer review. In
the influential Metric Tide report, a low agreement between metrics and peer
review in the UK Research Excellence Framework (REF) was found. However,
earlier studies observed much higher agreement between metrics and peer review
in the REF and argued in favour of using metrics. This shows that there is
considerable ambiguity in the discussion on agreement between metrics and peer
review. We provide clarity in this discussion by considering four important
points: (1) the level of aggregation of the analysis; (2) the use of either a
size-dependent or a size-independent perspective; (3) the suitability of
different measures of agreement; and (4) the uncertainty in peer review. In the
context of the REF, we argue that agreement between metrics and peer review
should be assessed at the institutional level rather than at the publication
level. Both a size-dependent and a size-independent perspective are relevant in
the REF. The interpretation of correlations may be problematic and as an
alternative we therefore use measures of agreement that are based on the
absolute or relative differences between metrics and peer review. To get an
idea of the uncertainty in peer review, we rely on a model to bootstrap peer
review outcomes. We conclude that particularly in Physics, Clinical Medicine,
and Public Health, metrics agree quite well with peer review and may offer an
alternative to peer review
An empirical analysis of the use of alphabetical authorship in scientific publishing
There are different ways in which the authors of a scientific publication can
determine the order in which their names are listed. Sometimes author names are
simply listed alphabetically. In other cases, authorship order is determined
based on the contribution authors have made to a publication.
Contribution-based authorship can facilitate proper credit assignment, for
instance by giving most credits to the first author. In the case of
alphabetical authorship, nothing can be inferred about the relative
contribution made by the different authors of a publication. In this paper, we
present an empirical analysis of the use of alphabetical authorship in
scientific publishing. Our analysis covers all fields of science. We find that
the use of alphabetical authorship is declining over time. In 2011, the authors
of less than 4% of all publications intentionally chose to list their names
alphabetically. The use of alphabetical authorship is most common in
mathematics, economics (including finance), and high energy physics. Also, the
use of alphabetical authorship is relatively more common in the case of
publications with either a small or a large number of authors
CitNetExplorer: A new software tool for analyzing and visualizing citation networks
We present CitNetExplorer, a new software tool for analyzing and visualizing
citation networks of scientific publications. CitNetExplorer can for instance
be used to study the development of a research field, to delineate the
literature on a research topic, and to support literature reviewing. We first
introduce the main concepts that need to be understood when working with
CitNetExplorer. We then demonstrate CitNetExplorer by using the tool to analyze
the scientometric literature and the literature on community detection in
networks. Finally, we discuss some technical details on the construction,
visualization, and analysis of citation networks in CitNetExplorer
Predicting the long-term citation impact of recent publications
A fundamental problem in citation analysis is the prediction of the long-term
citation impact of recent publications. We propose a model to predict a
probability distribution for the future number of citations of a publication.
Two predictors are used: The impact factor of the journal in which a
publication has appeared and the number of citations a publication has received
one year after its appearance. The proposed model is based on quantile
regression. We employ the model to predict the future number of citations of a
large set of publications in the field of physics. Our analysis shows that both
predictors (i.e., impact factor and early citations) contribute to the accurate
prediction of long-term citation impact. We also analytically study the
behavior of the quantile regression coefficients for high quantiles of the
distribution of citations. This is done by linking the quantile regression
approach to a quantile estimation technique from extreme value theory. Our work
provides insight into the influence of the impact factor and early citations on
the long-term citation impact of a publication, and it takes a step toward a
methodology that can be used to assess research institutions based on their
most recently published work.Comment: 17 pages, 17 figure
Field-normalized citation impact indicators and the choice of an appropriate counting method
Bibliometric studies often rely on field-normalized citation impact
indicators in order to make comparisons between scientific fields. We discuss
the connection between field normalization and the choice of a counting method
for handling publications with multiple co-authors. Our focus is on the choice
between full counting and fractional counting. Based on an extensive
theoretical and empirical analysis, we argue that properly field-normalized
results cannot be obtained when full counting is used. Fractional counting does
provide results that are properly field normalized. We therefore recommend the
use of fractional counting in bibliometric studies that require field
normalization, especially in studies at the level of countries and research
organizations. We also compare different variants of fractional counting. In
general, it seems best to use either the author-level or the address-level
variant of fractional counting
A smart local moving algorithm for large-scale modularity-based community detection
We introduce a new algorithm for modularity-based community detection in
large networks. The algorithm, which we refer to as a smart local moving
algorithm, takes advantage of a well-known local moving heuristic that is also
used by other algorithms. Compared with these other algorithms, our proposed
algorithm uses the local moving heuristic in a more sophisticated way. Based on
an analysis of a diverse set of networks, we show that our smart local moving
algorithm identifies community structures with higher modularity values than
other algorithms for large-scale modularity optimization, among which the
popular 'Louvain algorithm' introduced by Blondel et al. (2008). The
computational efficiency of our algorithm makes it possible to perform
community detection in networks with tens of millions of nodes and hundreds of
millions of edges. Our smart local moving algorithm also performs well in small
and medium-sized networks. In short computing times, it identifies community
structures with modularity values equally high as, or almost as high as, the
highest values reported in the literature, and sometimes even higher than the
highest values found in the literature
- …